Gustavo Diaz
Postdoctoral Fellow
Department of Political Science
McMaster University gustavodiaz.org
# Packageslibrary(tidyverse)
Warning: package 'tidyverse' was built under R version 4.2.3
Warning: package 'ggplot2' was built under R version 4.2.3
Warning: package 'tibble' was built under R version 4.2.3
Warning: package 'readr' was built under R version 4.2.3
Warning: package 'dplyr' was built under R version 4.2.3
Warning: package 'lubridate' was built under R version 4.2.3
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.1 ✔ readr 2.1.4
✔ forcats 1.0.0 ✔ stringr 1.5.0
✔ ggplot2 3.4.2 ✔ tibble 3.2.1
✔ lubridate 1.9.2 ✔ tidyr 1.3.0
✔ purrr 1.0.1
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(DeclareDesign)
Loading required package: randomizr
Loading required package: fabricatr
Loading required package: estimatr
Attaching package: 'DeclareDesign'
The following object is masked from 'package:dplyr':
vars
The following object is masked from 'package:ggplot2':
vars
library(kableExtra)
Attaching package: 'kableExtra'
The following object is masked from 'package:dplyr':
group_rows
# ggplot global optionstheme_set(theme_gray(base_size =20))
# got this through# https://automeris.io/WebPlotDigitizer/# And by reverse engineering table 1# Because replication data was not available# So this has some human errorvalidation =data.frame(technique =c("Direct question", "List experiment"),estimate =c(0.396, 0.487),upper =c(0.419, 0.562),lower =c(0.376, 0.417))actual_vote =0.65ggplot(validation) +aes(x = technique, y = estimate) +geom_point(size =4, alpha =0) +geom_linerange(aes(x = technique, ymin = lower, ymax = upper),linewidth =1, alpha =0) +ylim(0.2, 0.8) +labs(x ="Technique", y ="Estimated proportion of 'No' votes")
Validation
ggplot(validation) +aes(x = technique, y = estimate) +geom_hline(yintercept = actual_vote, linetype ="dashed",linewidth =1) +geom_point(size =4, alpha =0) +geom_linerange(aes(x = technique, ymin = lower, ymax = upper),linewidth =1, alpha =0) +annotate("text", x =0.7, y =0.7, size =5, label ="Actual vote share") +ylim(0.2, 0.8) +labs(x ="Technique", y ="Estimated proportion of 'No' votes")
Validation
ggplot(validation) +aes(x = technique, y = estimate) +geom_hline(yintercept = actual_vote, linetype ="dashed",linewidth =1) +geom_point(size =4) +geom_linerange(aes(x = technique, ymin = lower, ymax = upper),linewidth =1, alpha =0) +annotate("text", x =0.7, y =0.7, size =5, label ="Actual vote share") +ylim(0.2, 0.8) +labs(x ="Technique", y ="Estimated proportion of 'No' votes")
Validation
ggplot(validation) +aes(x = technique, y = estimate) +geom_hline(yintercept = actual_vote, linetype ="dashed",linewidth =1) +geom_point(size =4) +geom_linerange(aes(x = technique, ymin = lower, ymax = upper),linewidth =1) +annotate("text", x =0.7, y =0.7, size =5, label ="Actual vote share") +ylim(0.2, 0.8) +labs(x ="Technique", y ="Estimated proportion of 'No' votes")
List experiments are not always an improvement
Can we do better?
Double list experiment
List A
Californians for Disability (advocating for people with disabilities)
California National Organization for Women (advocating for women’s equality and empowerment)
American Family Association (advocating for pro-family values)
American Red Cross (humanitarian organization)
List B
American Legion (veterans service organization)
Equality California (gay and lesbian advocacy organization)
Tea Party Patriots (conservative group supporting lower taxes and limited government)
Salvation Army (charitable organization)
Sensitive item
Organization X (advocating for immigration reduction and measures against undocumented immigration)
Randomly appears in list A or B
Single list: Half of the respondents see sensitive item
Double list: Everyone sees it
Equivalent to making two parallel list experiments
I participated in activities organized by feminist groups
I went to religious ceremonies at the church in my neighborhood
Negatively correlated items
Things done in the last six months:
I saw people playing sports in my neighborhood
I visited friends on weekends
I participated in activities organized by feminist groups
I went to religious ceremonies at the church in my neighborhood
Positively correlated lists
Items paired by number:
List A
I saw people playing sports
I visited friends
I went to activities by feminist groups
I went to religious ceremonies
List B
I saw people playing soccer
I chatted with my friends
I went to activities by LGBTQ groups
I donated to charity at church
Randomized response
For this question, I want you to answer yes or no.
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw.
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no.
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no. If shows, tell me yes.
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no. If shows, tell me yes. But if another number shows, tell me your own opinion about the question.
[TURN AWAY FROM RESPONDENT]
Now you throw the dice so that I cannot see what comes out.
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no. If shows, tell me yes. But if another number shows, tell me your own opinion about the question.
[TURN AWAY FROM RESPONDENT]
Have you thrown the dice?
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no. If shows, tell me yes. But if another number shows, tell me your own opinion about the question.
[TURN AWAY FROM RESPONDENT]
Have you picked it up?
Randomized response
For this question, I want you to answer yes or no. But I want you to consider the number of your dice throw. If shows on the dice, tell me no. If shows, tell me yes. But if another number shows, tell me your own opinion about the question.
Now, during the height of the conflict in 2007 and 2008 (in Afghanistan), did you know any militants, like a family member, a friend, or someone you talked to on a regular basis?
Please, before you answer, take note of the number you rolled on the dice.